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Abstract 

This study used a Hierarchical Linear Modeling (HLM) approach to 
investigate relationships between student motivation, higher-order thinking 
skills, quality of teaching and learning, teacher student relations, student 
satisfaction with course contribution to their learning, and active learning 
strategies, with a sample of 2,190 undergraduate students. The HLM results 
indicate significant differences among departments in student motivation (2 
(31) = 260.90, p < 0.000). This variation among university departments 
suggests that the department-level variables might have accounted for the 
differences in motivation scores. An Intra-class correlation for this sample is 
0.17, indicating that 17% of variance in motivation was among departments. 
Further, the reliability of the sample mean in any department for the true 
mean department motivation was 0.82. All three level- 1 variables were found 
to vary among departments. Cross-level interactions showed that main effect 
of quality of teaching and learning was significant while student satisfaction 
with course did not provide significant contribution to their learning or to 
the explanation of the variation in motivation. The slopes for active learning 
strategies, higher-order thinking skills, and teacher student relations were 
all positive and significant. 

Background of the Study 

It is often common to hear faculty remark that today's college students 
seem less motivated to learn than those in years past, and as such it is critical 
that investigation is carried out to try to identify possible correlates and 
predictors of motivation of students in learning. Reform movements in various 



53 

subject areas of education such as mathematics promote the increased emphasis 
on higher-order thinking s kill s in the classroom (DiCintio & Stevens, 1997). 
National standards in many fields now place much more emphasis on critical 
analysis, problem solving, and integration of theories than has been typical in 
the classroom in these subjects for decades (Pascarella, Edison, Nora, Hagedorn 
& Braxton, 1996). Finding ways to increase student motivation in undergraduate 
courses has long been a challenge for faculty and administrators alike, 
particularly in large courses at large institutions. 

This study investigates whether classroom emphasis on higher-order 
thinking skills, teacher student relations, active learning strategies, quality of 
teaching and learning, and student satisfaction with course contribution to their 
learning are related to student motivation in learning. If such relationships are 
shown to be significant, faculty would be well advised to increase their focus 
on higher-order thinking skills among the other variables so as to further develop 
those skills in students and improve students' motivation in their classes. 

In most studies student motivation is considered as an independent variable 
in relation to cognitive skills such as higher-order thinking skills (Nastasi & 
Clements, 1994). If it can be established that student motivation can be 
effectively increased by doing what many educators would prefer to do anyway, 
then emphasis on such skills as critical analysis and integration of concepts by 
faculty, counselors, staff and education stakeholders might more likely change 
the way they carry out their responsibilities in the teaching, learning, and 
counseling environments. 

Conceptual/Theoretical Frameworks 

Motivation has been defined as the energy that is innate within all 
individuals, and high levels directed toward a particular situation results in 
greater amounts of energy expended on that task (McDevitt & Ormrod, 2004). 
Therefore, greater desire and higher levels of energy targeted at accomplishing 
a goal should result in higher levels of performance (House, 1997; McDevitt & 
Ormrod, 2004; Tavani & Losh, 2003). Goal theory of motivation maintains 
that the perceived purpose of a task is a critical factor in the quality of engagement 
in the task (DiCintio & Stevens, 1997). This theory describes two goal 
orientations, mastery goals and ego goals. Mastery goals are intrinsically 
motivated, self-referenced rather than based on normative comparison, and 
concerned with increasing competence through effort. Ego goals, in contrast, 
are extrinsically motivated with success determined in reference to others and 
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concerned with displaying competence for others to see. In middle school 
settings, a variety of factors in classroom and school environments have been 
shown to influence students' goal orientations as summarized by DiCintio 
and Stevens (1997). 

Kanfer and Ackerman (1989) noted that task difficulty affects the 
demonstration of ability-motivation interactions by changing the relationships 
among predictors and performance. For less difficult tasks, ability may be 
less powerful as a predictor of performance than motivation. For more difficult 
tasks, however, performance is likely to be significantly affected by the 
interaction effects of motivation and ability. Kanfer and Ackerman (1989) 
considered the construct of cognitive resources or attentional resources to be 
a link between ability and motivation. Most research findings on motivation 
have concluded that academic performances and motivations are significantly 
related to one another. 

It is important to note that teachers, through their roles are an influencing 
agent for student motivation. For instance, encouraging students in their pursuit 
for excellence in learning, providing positive feedback, being involved in 
positive interactions, remaining enthusiastic about students and student 
educational growth, and cultivating a positive classroom environment, have 
a strong impact on student academic motivation (Astin, 1993; Bean & Kuh, 
1984; Juarez, 2001; Lamport, 1993). 

Justification for the Study 

Motivation theory has been discussed as an important aspect of student 
success in schools. Research has shown that that motivation influences 
students' involvement and academic achievement (Gambrell, 2001). There 
also is a growing interest in understanding the relationships between 
motivation and teacher- student relationship. This study seeks to investigate 
if teacher- student relations influence motivation when students' higher order 
thinking skills and active learning strategies are taken into account. 

Being aware of how teacher- student interaction (relations) can promote 
academic motivation may provide implications in a variety of areas for 
educators. Teachers can likely restructure the teaching and learning 
environment by providing different learning strategies to students and finding 
ways to motivate students to learn and to engage them in active learning. 
Without knowing and understanding how teacher- student relations influence 
motivation, teachers may limit their abilities to improve instruction. 
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Students engage in learning through behaviors and motivation and those 
learners who are highly motivated remain engaged, enthusiastic and are 
more likely to participate in academic activities. On the contrary, the less 
motivated the students are, the less they remain engaged in learning. 

Most previous studies have not directly examined the impact of emphasis 
on higher-order thinking skills, teacher student relations, and active learning 
strategies on student motivation and involvement in the university classroom 
(DiCintio & Stevens, 1997; Nastasi & Clements, 1994). This study will 
investigate the direction and magnitude of these relationships for a sample 
of undergraduates at a major public university. Most prior studies of the 
relationship between these variables used much smaller samples than the 
2,190 subjects in this study, thereby limiting the statistical power of those 
smaller studies. Many of these studies also focused on elementary and middle 
school higher-order thinking skills, which are both qualitatively and 
quantitatively different from the higher-order thinking skills typical at the 
college level. If indeed emphasis on higher-order thinking skills is shown 
to predict student motivation and involvement, then college and university 
faculty would have greater incentive to change their teaching methods to 
increase their emphasis on higher-order thinking skills. 

Further, much of prior research considered motivation in young children 
in relationship to cognitive skills that are developmentally appropriate for 
elementary and middle schools (DiCintio & Stevens, 1997; Nastasi & 
Clements, 1994; Singh & Singh, 1994). This study looks directly at whether 
student motivation is related to the emphasis on higher-order thinking skills, 
active learning strategies, quality of teaching and learning, and student 
satisfaction with course contribution to their learning in a post-secondary 
teaching and learning environment, and specifically in the university 
classroom. 

Whereas prior studies have considered the student's motivation as the 
independent variable and cognitive skills including higher-order thinking 
skills as dependent variables, this study considers whether the emphasis on 
higher-order thinking skills in relation to active learning strategies and 
teacher student relations affect student motivation. Also, earlier studies 
measured actual performance on tests of higher-order thinking rather than 
the emphasis given to those skills in the classroom (Kanfer & Ackerman, 
1989). In light of the aforementioned statements, this study posits that 
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emphasis on higher-order thinking skills in the classroom, teacher student 
relations, and active learning strategies will serve to motivate students and 
increase their involvement in the class and directly or indirectly lead to 
positive academic change. 

Pertinent Research 

Motivation and Higher-Order Thinking Skills 

The literature regarding the relationship between motivation and higher- 
order thinking skills is drawn from diverse settings such as K-12 education, 
higher education, and even training to become military air- traffic controllers. 
In few cases, motivation is viewed as the dependent variable with higher- 
order thinking skills as an independent variable. More often, motivation is 
viewed as a predictor of higher-order thinking skills. 

Nastasi and Clements (1994) studied motivation and higher-order thinking 
in third-grade students in two cooperative computer environments, one using 
Logo for programming and the other using computer-based instruction for 
writing. They found that the treatment (environment) accounted for significant 
portions of the variance in both motivation and in higher-order thinking skills 
and that motivation accounted for a significant portion of the variance in 
higher-order thinking skills. While this study treated higher-order thinking 
skills as the dependent variable, the authors maintained that the learning 
environment that placed greater emphasis on higher-order thinking skills saw 
higher levels of student motivation. 

DiCintio and Stevens (1997) investigated whether the level of higher- 
order thinking required by instruction is related to motivational goals in middle 
school mathematics classrooms. Higher-order thinking skills were found to 
be a significant predictor of motivational variables with fifth-grade providing 
more emphasis on higher-order thinking skills than sixth- and seventh-grade 
mathematics classes and much more emphasis on higher-order thinking skills 
in a high-ability fifth-grade class than in an average ability fifth-grade class. 
The motivational variable of mastery orientation (intrinsic motivation) was 
also significantly higher for fifth-graders than for the older students in the 
study. The authors, however, cited a lack of variance in higher-order thinking 
skills within grade level as a confounding issue limiting generalization of the 
study's conclusions. 
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Kanfer and Ackerman (1989) conducted a study of skill acquisition in 
U.S. air force recruits learning to become air- traffic controllers. Their research 
found that goal assignments provided during the declarative stage of complex 
skill acquisition actually decreased performance among both low-ability and 
high-ability groups with more impairment among low-ability subjects. They 
define the declarative phase as the earliest phase of skill acquisition when the 
focus is on facts. This stage is followed by stages of integration and 
automatization of the skill. Higher-order thinking skills are most closely related 
to the integration phase of skill acquisition. The study concludes that low- 
ability subjects benefit more from imposition of a goal assignment (motivation) 
during the integration portion of complex skill acquisition than do high-ability 
subjects. Their conclusion can be stated another way, that emphasis on higher- 
order thinking skills provides an environment in which motivational 
interventions are more effective. While learning to become an air- traffic 
controller certainly differs from academic learning in a university setting, 
Kanfer and Ackerman's (1989) study is one of few to consider the effect of 
emphasis on higher-order thinking skills on motivation rather than vice versa. 

Other studies that have investigated the relationship between motivation 
and higher-order thinking skills with mixed results include research by Singh 
and Singh (1994) on the role of motivation in integrational capacity in young 
children. They found that motivation was a significantly better predictor of 
integrational capacity than an ability construct was for some but not all age 
levels. Mayer (1998) found that motivational skills were necessary in academic 
problem solving but cognitive and strategic (metacognitive) skills were even 
more necessary. House (1995) found that motivational variables were 
significant predictors of grade performance but that academic background 
was a stronger predictor. Prediction models presented in these studies varied 
by ethnicity and by gender. 

Motivation and Teacher Student Relations 

Kuh and Hu (2001) stressed that meaningful and frequent interaction 
between teachers and students are important to learning. While recognizing 
the importance of the teacher student interactions, Astin (1993), Bean and 
Kuh (1984), Juarez (2001), Lamport (1993), and Kuh, Douglas, Lurd, and 
Ramin (1994) investigated the relationships between faculty and students 
and how teacher-student relations influences motivation and academic 
performance. Astin (1993) found that quality of teacher interactions have a 
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positive association with student academic outcomes while Pascarella and 
Terenzini (1991) reported that student-faculty interaction is an important factor 
in college student development. If a positive teacher- student relationship 
brings positive impact on students' motivation and achievement, it is possible 
to consider that negative teacher- student relationship will probably cause 
negative impact on motivation on achievement. Plecha (2002), however, found 
no significant difference in negative impact on students' motivation and 
achievement based on negative teacher- student relationship. 

Teacher as a motivation factor influence students' learning greatly. How 
teachers behave in classroom can directly promote student learning and 
motivation. It is important that teachers care about how they motivate students 
because if the students are not actively motivated to learn, learning will not 
occur. Patrick, Hisley, Kempler, and College (2000), noted that teacher 
behaviors promote student intrinsic motivation to learn. Their research of 
how teachers' enthusiasm and interaction relate to each other found that a 
significant relationship between teacher-student relations and intrinsic 
motivation. Students who are intrinsically motivated view learning as a goal 
in itself, while students who are extrinsically motivated view learning as a 
reward (Cokley, 2000; Covino, & Iwanicki, 1996; Dweck, 1986). 

Teacher Student Relationships 

Tinto's model (1987) posits that persistence is largely determined by 
the levels of academic and social integration that a student experiences with 
the institution. Tinto is also credited for developing a model of factors crucial 
to academic continuation which includes student-teacher relationship. Tinto 
(1987, 1993) as well as Woodside (1999) concluded that both informal and 
formal interaction with teachers is important in predicating freshmen academic 
outcomes, satisfaction, and attrition. 

Students learn how experts (mentors) think and solve problems by 
interacting with faculty members inside and outside the classroom 
(Institutional Benchmark Report, 2002). Numerous projects have focused on 
the relationship that exists between student-faculty interactions and outcome 
variables such as academic achievement and overall satisfaction of college 
students (Astin, 1993; Kuh, 2001; Terenzini & Pascarella, 1994). Factors 
associated with student-faculty interaction include; student's academic 
achievements, educational attainment and aspirations, career and major choice, 
college satisfaction and persistence, and cognitive development. First-year 
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students have occasional contact (once or twice a month) with their instructors, 
while seniors at doctoral-extensive universities interacted less with faculty 
members than first-year students at liberal arts colleges, (Kuh, 2001). 

Faculty exerts much influence in their out-of-class as well as in-class 
contact with students. Terenzini and Pascarella (1994) pointed out that faculty 
educational influence is enhanced when their contacts with students extend 
beyond the formal classroom to the informal non-classroom setting. The 
extent of student informal contact with faculty is positively linked with a 
wide array of outcomes— perception of intellectual growth during college, 
increase in intellectual orientation and curiosity, liberalization of social and 
political values, growth in autonomy and independence, increase in 
interpersonal skills, and educational aspiration, persistence and attainment 
(Terenzini & Pascarella, 1994). 

Astin (1993) found that student-faculty interaction had a significant 
positive correlation with every academic attainment outcome such as college 
GPA, degree attainment, graduating in honors and enrollment in graduate 
or professional school. Further, Astin stated that student-faculty interaction 
is associated with student satisfaction with quality of instruction, support 
services, intellectual and personal growth, behavioral outcomes, career 
outcome and the overall college experience. Pascarella and Terenzini (1976, 
1991) pointed out that the frequency of student-faculty interactions 
significantly predicts freshman academic outcomes such as college 
satisfaction and attrition. 

Student-faculty interaction produces a sense of identification with faculty 
and has important implications for student development (Astin 1993). 
Pascarella and Terenzini (1991) noted that the change in student's intellectual 
orientation is influenced by interaction with people in the college setting- 
faculty and students. Tinto (1987, 1993) stated that student-faculty 
interactions, which include both formal classroom experiences and informal 
interactions out of class, are crucial to the academic continuation and 
intellectual development of students. According to Tinto (1987, 1993), a 
lack of such interactions is a significant determinant of attrition. One project 
that specifically examined the relation between student-faculty interaction 
and academic performance found that student-faculty interactions had a 
significant influence on students' academic performance as measured by 
students' SAT scores and freshman year cumulative GPA. The interactions 
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were most powerful in affecting achievement if they concerned intellectual 
or course-related subjects. It was also found that students who interacted 
more frequently with faculty performed better academically than what was 
predicted from their SAT scores. On the other hand, students who seldom 
met with faculty tended to achieve at lower levels than predicted. Taken 
together, the existing research suggests that student-faculty interactions are 
important to a student's college experience (Astin, 1993; Pascarella & 
Terenzini, 1991; Woodside, 1999). 

Quality of Teaching and Learning 

Grade point average provides only one indicator of educational quality 
and may or may not truly reflect what a student gains from a classroom 
experience. As colleges increasingly focus on developing students' critical 
thinking skills, measures of the impact of college on development of these 
skills proliferate. For adult students, with a few years in the workforce prior 
to returning to school, we may find an inflated positive impact of work on 
cognitive growth. Pascarella (2001) concluded from his numerous studies 
that focus on traditional undergraduates that particularly during a student's 
later years in college "work has a greater chance of being focused and 
integrated with a student's academic program and career goals later in his or 
her academic career" (p. 22). The same positive effect may occur in adult 
students who view the value of higher education through their own unique 
experiences and choose to return to school not because of societal expectations 
but because they see a personal or professional value the additional education 
may provide. 

Teacher encouragement and verbal praise are one of key factors in student 
academic motivation (Hancock, 2000). An encouraging teacher can provide 
a student with the essential link between school and the home community. 
Cokley's (2000) research on the impact of how faculty encouragement affects 
student academic motivation, found that students had higher academic 
motivation when faculty encouraged them. Caring teacher-student relationship 
improved motivation according to Juarez (2001). Because students perceived 
teachers' caring as having support and confidence from their teachers, they 
were motivated more. When talking about teachers' instructional method with 
motivation. Flowers, Hancock, and Joyner's (1999, 2000) investigation on 
teachers' instruction method and motivation found that teachers' direct and 



Journal of Educational Research & Policy Studies 



61 

indirect instructional methods is a factor for students' motivation to learn 
academic content. Further, teachers are responsible for creating an 
environment that the students could become motivationally active in learning. 
Findings from various studies have found that to motivate students' learning 
it is imperative to provide positive classroom environment (Bembenutty, 
McKeachie, Karabenick, & Lin, 2001). Ames (1992) reported that a positive 
relationship between the classroom environment and students' intrinsic 
motivation. Whether teachers are supportive or controlling influences the 
structure of the classroom, and thus affects student motivation to learn and 
involvement in classroom activities. 

Active Learning Strategies 

The importance of student involvement or engagement and quality of 
effort as a significant determinant of students' educational outcomes is an 
important factor in predicting student academic achievement (Astin, 1984; 
Pace, 1984). "Active learning invites students to bring their life experiences 
into the learning process, reflect on their own and others perspectives as they 
expand their viewpoints, and apply new understanding to their own lives" 
(ACPA & NASPA, 1997, p. 3). Students tend to learn more when they are 
actively involved in their education in different settings (Institution Benchmark 
Report, 2002). In addition, collaborating with other students in solving 
problems or mastering difficult materials prepares the student for problems 
they will encounter during and after college (Institution Benchmark Report, 
2002). Motivating, inspiring, and teaching students to assume responsibility 
in their education process enhances productive learning (Kuh, Lurd, & Ramin, 
1994). Kuh (2001) noted that a higher proportion of students are involved in 
active, collaborative, and service learning. Further, Kuh points out that more 
than 90% of the students work with other students on projects during class 
and 63% of seniors engage in community service or volunteer work. 

Purpose of the Study 

The purpose of this study is to investigate whether student motivation is 
related to the amount of perceived emphasis on higher-order thinking skills, 
active learning strategies, and teacher student relations for college students 
within individuals and across departments. The focus is to examine factors 
within and across college/university departments that college instructors, 
administrators, counselors and support staff can influence to increase student 
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motivation and thus student success. Specifically, this study is guided by 
four research questions: a) Are there differences in motivation scores among 
university departments? b) Do department level factors (quality of teaching 
and learning, student satisfaction with course contribution to their learning) 
explain the differences in mean department motivation scores? c) Do student 
factors (higher-order thinking skills, active learning strategies, and teacher 
student relations) explain differences in motivation scores? d) Do department 
level factors influence the magnitude of the student factors on motivation 
scores? 

Methodology 
Research Design 

This study was based on multiple design elements. It was cross-sectional 
in that it considered variables of a specific higher education institution's 
teaching and learning environment at a specific point in time. It also used a 
survey design to collect information from students on the key variables related 
to teaching and learning. 

Internal and External Validity 

Internal validity helps determine if the predictor variables really influenced 
the level- 1 and level-2 outcomes variables. Threats to the internal validity 
offer competing explanations for the relationships. Although threats to internal 
validity are present in every study, the author attempted to minimize the factors 
that might control, influence or distort the study results. Since a single survey 
was administered, the researcher could control for history or the confounding 
effect of specific events that occurs between pretest and posttest. However, 
we cannot know if the survey respondents did indeed engage in conversations 
when completing the survey. Likewise, the single-survey research design 
allowed the author to control for the most common threats to internal validity 
in survey: location, maturation, attrition or experimental mortality, and any 
testing effects, although the length of the questionnaire and inclusion of other 
subscales not relevant to the current study could have impacted the 
respondents' perceptions of the key variable of this study. 

Some challenges to internal validity of this study remain. Selection-history 
effects were a concern. Since there was no random assignment in this quasi- 
experimental study, there may be differences between those who work full- 
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time and those who do not work other than those discussed in this study since 
the respondents were both traditional and non- traditional learners. Previous 
research found some other academic characteristics and behaviors correlated 
to undergraduate academic performance (Marlowe, Koonce, Lee, & Cai, 
2002). These characteristics may include age, economic differences, 
differences in family responsibilities, ACT or SAT scores, academic major, 
transfer versus non-transfer, or even where students sit in class. Additional 
factors include learning styles, personalities, or involvement in other 
extracurricular activities. The author could not control for any of these 
confounding variables. Similarly, the non-random nature of the research 
project could cause students to self-select the evening program based on 
characteristics unrelated to work status, but that could impact their perceptions 
of development of the key variables of this study, especially higher order 
thinking skills (HOTS) or academic effort. For example, previously cited 
research demonstrates that lower income students tend to delay college 
enrollment until after the age of 25 (King, 2002). The sample used in this 
study could over-represent lower income students, although the impact of 
such over-representation on our study outcome remains uncertain. 

With the external validity issue, concern in this case lies with the types of 
subjects tested, the specific setting in an evening program, and the time in 
history when data collection commenced. Because of this sample, that is, 
evening school students at a large southern public university, the findings 
may not be generalizable to other types of program models, private colleges, 
or colleges outside of the southern public university. 

Measures 

A variety of self-report measures have been developed to examine student 
perception of learning environment and their own characteristics as learners. 
This study used measures contained in Student Assessment of Teaching and 
Learning (SATL) (Short-Form), first developed by Ellett, Culross, McMullen, 
and Rugutt, (1996), and later revised by Ellett, Loup, Culross, McMullen and 
Rugutt (1997). The measures assessed a wide variety of factors among college 
students. Student motivation and involvement (MI), is comprised of the sum 
of factor scores for five statements. Students respond to the degree to which 
various activities enhanced their learning using a three point scale as follows: 
1 = learning not enhanced, 2 = learning sometimes enhanced, 3 = learning 
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almost always enhanced. A complete list of the five statements making up 
the MI construct is found in Appendix A. 

The variable for classroom emphasis on higher order thinking skills 
(HOTS) is comprised of the sum of factor scores for five statements (See 
Appendix A). Students rate the amount of emphasis given to each type of 
learning as follows: 1 = no emphasis, 2 = some emphasis, 3 = much emphasis, 
4 = very much emphasis. 

The quality of teaching and learning (QTL) construct was computed as 
the sum of responses to 22 statements (see Appendix A). Students respond to 
the degree to which various activities enhanced their learning using a three 
point scale as follows: 1 = learning not enhanced, 2 = learning sometimes 
enhanced, 3 = learning almost always enhanced. 

The active learning strategies (ALS) and teacher student relations 
constructs were generated from a 52 item survey using a five-point forced 
Likert- type scale ranging from almost never to almost always. ALS construct 
was the sum of scores for 15 items while the teacher student relations (TSR) 
construct was the sum of scores on six items (See Appendix A). The last 
construct was the sum of three survey items where students were asked to 
make summative judgements about several course-related factors such as to 
grade the quality of teaching in the course, to rate the course in terms of its 
contribution to their personal learning, and to arrive at an overall course grade 
using a 100-point scale (see Appendix A). 

The subscales of the instrument, items per scale and their corresponding 
Cronbach alpha reliabilities (see Table 1) are: Quality of Teaching and 
Learning (QTL, 0.95), Active Learning Strategies (ALS, 0.95), Higher-Order 
Thinking Skills (MI, 0.83), Teacher Student Relations (TSR, 0.92), Student 
Satisfaction with Course contributions to their Learning (SATISFY, 0.88) 
and Motivational Involvement (HOTS, 0.86). Definitions of constructs are 
provided in Appendix B. 

Sampling 

The sample for this study consisted of 2,190 students from 145 classes in the 
Evening School of the Division of Continuing Education at one large southern 
university. During the semester, the sampled students took a variety of courses 
in such topic areas as mathematics, natural science, social science, and 
humanities. They also represent a broad array of individuals, including 
traditional-aged, nontraditional-aged, differing employment status, and gender. 
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This sample was 40% male, 60% female; 60% not employed full-time; 69% 
were traditional students while 31% were non-traditional students (see 
Table 2). 

Further, the final useable data for the Hierarchical Linear Modeling 
(HLM) analysis consist of 1,633 observations collected from 32 University 
departments from both traditional and non-traditional students. From the HLM 
dataset, students who had missing data for gender, and traditional/non- 
traditional variables were excluded from the study. Other variables that had 
fewer missing data (less than 1%) had mean substitution performed. 

Table 1 



Cronbach Alpha Reliability Coefficients 



Variable 


Variable Description 


Alpha Reliabilities 


QTL (22)* 


Quality of Teaching and Learning 


.95 


Ml (5) 


Motivation 


.86 


TSR (4) 


Teacher Student Relations 


.92 


HOTS (5) 


Higher -Order Thinking Skills 


.83 


ALS (15) 


Active Learning Strategies 


.95 


SATISFY (3) 


Student Satisfaction with Course 
Contribution to their Learning 


.88 


Note. * Number of items comprising measure. 




Table 2 






Demographic Distributions of the Study Sample 






A* 


Percent (%) 


Gender 






Male 


859 


39.75 


Female 


1,302 


60.25 


Work full time 






Yes 


864 


39.69 


No 


1,313 


60.31 


Traditional 






Traditional 


1,117 


68.87 


Non-traditional 


505 


31.13 



Note. * Total for each demographic variable may not add up to 2,190 due to non-responses. 
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Data Analysis 

Data analyses completed included: a) descriptive statistics for 
characteristics of the sample and the various measures, b) factor analysis, 

c) Alpha internal consistency reliabilities for measurement subscales, and 

d) Hierarchical Linear Modeling (HLM). The HLM approach is discussed in 
detail and its associated results presented. 

The HLM model was used because of the nature of the research questions 
posed, and the need for simultaneous evaluation of student-level, and 
department-level variables, and the hierarchical nature of the data. The choice 
of HLM statistical strategy was done on the basis that students are grouped 
or nested within university departments, and standard regression analysis 
will result in smaller standard errors and possibly incorrect conclusions. 
Analysis of department-level variables at the individual student level will 
result in aggregation bias, while aggregating student-level data to the group 
level fails to fully capture the effects of outcome variables that may operate 
at both levels of analysis (Fullerton, 2002; Hox 2002; Snijders, & Bosker, 
2000). Hierarchical linear modeling techniques allow researchers to model 
individual outcomes within groups and then to identify and model any 
between-group differences that occur (Bryk & Raudenbush, 1992; Hox, 2002; 
Raudenbush & Bryk, 2002). 

Model Specification 

The two-level hierarchical linear models were specified and analyzed 
employing full maximum likelihood estimation using HLM 5.0 (Raudenbush, 
Bryk, & Congdon, 2001). 

The researcher presents the two-level conditional models using notation 
consistent with that used by Bryk and Raudenbush (1992) and Raudenbush 
and Bryk (2002). To give contextual meaning to the models, in this study 
they are referred to as a system where the data structure is students in university 
departments. The basic model structure and notation, however, can be applied 
to any conditional two-level system. 

Model 1 

Model 1, which includes no predictors at the student level, partitions 
the total variance in motivation score into its within- and between department 
components. The department-level residual value from this model is used as 
an indicator of unadjusted department average motivation score (see Equations 
1 and 2). 
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In this study, a basic level-2 linear model, which served as a baseline is 
discussed. The outcome variable of interest is the result of motivation score, 
represented by the variable motivation (MOTIVATION). 

MOTIVATION (7 ) = d . + r . (1) 

Using this data, the level- 1 coefficient becomes an outcome variables at 
level-2 (in this case the 32 units). The intercept is modeled as varying randomly 
across the departments. 

% = % + 

( 2 ) 

It is assumed that the level- 1 error term, denoted by r., is normally 
distributed with a mean of zero and a constant variance 6^. Equations 1 and 2 
are the unconditional models developed to provide the baseline data for 
comparisons with subsequent conditional models which have model 
predictors. The unconditional level- 1 model, predicts the outcome within each 
level- 1 unit with just one level-2 parameter, d^undicates the mean outcome. 
On level 2 of the model, represents the mean outcome over departments 
and u^.is a random term assumed to be normally distributed with a constant 
variance. 

Model 2 

Model 2 adds student level predictors by regressing motivation for 
student i within department j on active learning strategies, higher-order 
thinking skills, and teacher- student relations. The level 1 model (student level) 
is 

Level 1 - Hierarchical Linear Model for Individual Student 

MOTIVATION (7.) = d^. + dfALS).. + d^/HOTS) .. -r d3/TSR) .. + r. (3) 

Where: 

MOTIVATION (7.) = Motivation score of student i in department j; 

The four parameters in the model may be interpreted as follows: 

d^. = mean motivation score (MOTIVATION) in department j, 

d^. = differentiating effect of active learning strategies (ALS) 
in department j, 

d = differentiating effect of higher-order thinking skills (HOTS) 

AJ 

in department j, 
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d^. = differentiating effect teacher student relations (TSR) in 
department j, and 

r.. = is a random "student effect", that is, the deviation of 

y 

Student i in department j from the department mean. 

Level 1 predictors are department-mean centered so that the intercept d^., 
can be interpreted as adjusted mean motivation score for department j. This 
adjustment is chosen to sort out the unique effects of department motivation 
score after controlling for the influences of student personal characteristics. 

At Level 2 the researcher modeled the intercept, mean motivation score 
in department j, and the three level- 1 regression coefficients as a function of 
department level characteristics. Because the student level predictors are 
centered around the department mean, the intercept is the department- 
mean outcome while d^^ is the average of the department means on motivation 
across the population of departments. 

Level 2 - Department-level Model 

% = < + «oi(QTL), + «o2(SATISFY)^, + u. 

ALS(d^p = d^^ + d^PQTL). -h d^^(SATISFY). -h u^. 

HOTS(dp = d^^ + d^PQTL). -h d, ^(SATISFY), -h u^. 

TSR(fl3p = -h d3PQTL), -h «3, (SATISFY), -h u^. (4) 

Equation 4 can further be summarized such that the coefficients d.^. from 
the level- 1 model can be modeled as outcome variables for level-2 model. 
For example: 

% = <+ «o,(QTL), + (SATISFY)^ + 

d^. = mean motivation score in department j, 

dpQ = the average of the department motivation score 
across the population of departments. 

dpj = differentiating effect of mean department quality 
of teaching and learning (QTL) in department j, 

= differentiating effect of mean department student 
satisfaction with course contribution to their learning 
(SATISFY) in department j, and 
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u^. = is a level-2 random effect that represents the deviation 
of department fs level- 2 coefficient, a. from its 
predicted value based on the department-level model. 

The slope coefficients can also be interpreted in the same way, for example; 
(HOTS) d = differentiating effect of higher-order thinking skills on motivation 
regression slope across departments. 

<33 j = differentiating effect of quality of teaching and learning on teacher student 
relations toward motivation. 

Results 

The results are presented by research question. Table 1 contains Cronbach 
alpha reliability coefficients for all study constructs. Table 2 presents 
demographic information for the study sample. The HLM results for all 
research questions are presented in Tables 3, 4, 5, and 6. Table 3 presents 
results from the unconditional random coefficient model for individual 
students. Table 4 provides HLM random Level- 1 reliability coefficients. Table 
5 presents HLM coefficients and standard errors, and Table 6 presents HLM 
variance components and standard deviation for level 2 predictors. Complete 
results of the study are presented in the following section. 

Research Question 1 : Are there differences in motivation scores among 
university departments? 

The results in Table 3 indicate that there were significant differences among 
departments in student motivation. The test statistic = 260.90, 31), 

indicates significant (p < 0.000) variation among university departments in 
their motivation scores and suggests that the department-level variables might 
have accounted for the differences in motivation scores. The grand mean 
motivation score is 84.85. Table 3 also provides the maximum likelihood 
estimate of the variance components. At the student level, the variance 
components is 6^ = 247.84. This indicates that students were more variable in 
their motivation scores. At the department level, 6^^ was the variance of the 
true department means, around grand mean d^. The variance components 
for department means was 6^^ = 50.33. To gauge the magnitude of the variation 
among departments in their mean motivation levels, a 95% confidence interval 
was computed for this sample. For the study sample, with mean of 84.85 and 
a standard error of 1.38, would indicate a 95% confidence interval of 84.85 ± 
1.96 (1.38) = (82.15, 87.55). 
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The coefficient of 50.33 with corresponding chi-square value of 260.90, 
indicate some variability among departments in terms of their motivation 
scores. The largest variance component is at level- 1 of the model for the 
study sample (247.84) presented in the preceding section), indicating that 
quite a lot of the variation in motivation remains unexplained by unconditional 
model presented in equation 1 . 

The unconditional model (Equation 1) also provides the necessary 
information to calculate the intraclass correlation coefficient. The intraclass 
correlation is the proportion of the total variation that is accounted for by the 
variation between level-2 units. It represents the proportion of variance in 
motivation scores between departments. This is calculated as an intra-class 
correlation for this sample is 0. 169 (17%). This intra-class correlation showed 




P = % + 62 (5) 

some proportion of variation among departments, indicating that 17% of 
variance in motivation was among departments. Table 3 presents results from 
the unconditional random coefficient model. 



Table 3 



Results from the Unconditional Random Coefficient Model for 
Individual Students (n = 1,633) 



Fixed Effects 
Total Sample 


Coefficient se"" 


df 


t 


P 


Average Department Mean, d^. 


84.85 1.38 


31 


61.36 


0.000 


Random Effect 
Total Sample 


Variance 

Components 


df 




P 


Department mean, u^. 
Level- 1 effect, r.. 

^ — V 


50.33 

247.84 


31 


260.90 


0.000 



Note: “Standard error; '’degrees of freedom. 



The HLM program, in its standard output provides the reliability of the 
least square estimated coefficients. These reliabilities can be estimated at 
both levels 2 and 3 of the model. For the departments (level-2 units), the 
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estimated reliability is defined as the ratio between the level-2 variance 
component and the sum of the level-2 and level- 1 components, with the latter 
divided by the number of observations within that particular cluster. Further, 
an estimator of the reliability of the sample mean in any department for the 
true mean department for this study was 0.823, for the unconditional model. 
This reliability coefficient indicates that the sample means were quite reliable 
as indicators of the true department means. Reliability coefficients were also 
computed for the conditional model. 

The HLM reliability coefficients provided in Table 4 are overall or average 
reliability for each level- 1 coefficient across the set of level-2 units 
(departments). These estimates depend on two factors: the degree to which 
the true underlying parameters vary from group (department) to group and 
the precision with which each group's regression equation is estimated (Bryk 
& Raudenbush, 1992; Raudenbush & Bryk, 2002). The precision of estimation 
of the intercept (department mean) depends on the sample size within each 
department, while the precision of estimation of the slope depends both on 
the sample size and on the variability of the model predictors within that 
department. The groups (departments) that are homogeneous with respect to 
the model predictors will exhibit slope estimation with poor precision (Bryk 
& Raudenbush, 1992; Raudenbush & Bryk, 2002). 

The conditional model reliability coefficient for the intercept was 0.65 1 
(see Table 4). Further, active learning strategies and higher-order thinking 
skills slope reliability coefficients were 0.308, and 0.139, respectively. This 
indicates that the model reliability coefficients for the intercepts, active 
learning strategies, and higher-order thinking skills randomly varying slopes 
had moderate reliability coefficients. The slope estimates for teacher student 
relations was far less reliable (0.019) than the slope intercept estimates. The 
primary reason for relatively low reliability coefficients for some slopes is 
that the true slopes variances for the model predictors across departments are 
much smaller than the variances of the true means (intercepts). Also, the 
slopes are estimated with less precision than are the means because many 
departments are relatively homogeneous on some of the model predictors. 
Bryk and Raudenbush (1992) stated, however, that coefficient reliabilities 
above 0.05 are acceptable. Based on this, the model reliability coefficients 
for intercepts, active learning strategies, and higher-order thinking skills were 
within acceptable limits (see Table 4 for details). 
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Table 4 



Hierarchical Linear Model Random Level-1 Reliability Coefficients for 
Study Sample {N = 1,633) 



Random Level- 1 Reliability Coefficients 


Reliability 

Coefficients 


Department mean (Intercept) 


0.651 


Active learning strategies 


0.308 


Higher-order thinking skills 


0.139 


Teacher student relations 


0.019 



Research Question 2: Do department factors (quality of teaching and 
learning and student satisfaction with course contribution to their learning) 
explain the differences in mean department motivation scores? 

The group-level variables that are observed to be significantly related to 
the random coefficients are termed cross-level interactions (Bryk & 
Raudenbush, 1992; Raudenbush & Bryk, 2002; Snijders & Bosker, 2000). 
Cross-level interactions for this study simply mean that a department level 
variable influences a student-level slope. All Level- 1 variables (active learning 
strategies, higher-order thinking skills, and teacher student relations) were 
found to vary among departments. The three coefficients were then modeled 
as department level variables; that is, each randomly varying coefficient 
became a model. Table 5 shows the final estimates of the fixed effects. From 
these results, main effects (department means-intercepts) for quality of 
teaching and learning provided significant contribution to the explanation of 
the variation in motivation while student satisfaction with course contribution 
to their learning did not. The slopes for active learning strategies, higher- 
order thinking skills, and teacher student relations were positive and 
significant. Full cross-level interactions between department- and student- 
level for the model outcomes and predictors are presented in Table 5. Here, 
in explaining the intercept, level-2 variables (e.g., quality of teaching 
and learning) leads to a main effect of quality of teaching and learning, while 
explaining the coefficient d^. of, say, active learning strategies, by level-2 
variable student satisfaction with course contributions to their learning leads 
to a product interaction effect of active learning strategies and student 
satisfaction with course contributions to their learning. 
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Hierarchical Linear Model Coefficients and Standard errors for Study 
Sample 



Study Sample 



Dependent Variable 


Predictor 


coefficient 


standard error 


Intercept: Department 


Intercept 


85.35* 


0.70 


Motivation mean 


QTL 


1.22* 


0.28 




Satisfy 


0.06 


0.29 


Slope: Average ALS- 


Intercept 


0.22* 


0.03 


Motivation regression slope 


QTL 


0.02 


O.OI 


across departments 


Satisfy 


-0.02 


O.OI 


Slope: Average HOTS- 


Intercept 


0.27* 


0.02 


Motivation regression slope 


QTL 


-0.02 


O.OI 


across departments 


Satisfy 


0.02 


O.OI 


Slope: Average TSR- 


Intercept 


0.14* 


0.02 


Motivation regression slope 


QTL 


-O.OI 


O.OI 


across departments 


Satisfy 


-O.OI 


O.OI 



Note: * /? < 0.05; 

QTL: Quality of Teaching and Learning 
HOTS: Higher-rder Thinking S ki lls 
ALS: Active Learning Strategies 
TSR: Teacher Student Relations 

Satisfy: Student Satisfaction with Course Contribution to their Learning 

Research Question 3: Do student factors (active learning strategies, higher- 
order thinking skills, and teacher student relations) explain differences in 
motivation scores? 

Estimating Explained Variance in HLM Models 

The analyses completed here followed the Raudenbush and Bryk (2002), 
where the explained variance at level- 1 reflects the proportional reduction in 
individual variation at level- 1 given the level-2 model, that is, the explained 
variance at level-2. The process is accomplished through setting up two 
models, one with no level-2 predictors and the final, which is the random 
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intercept (or slope) variance model estimated from the final fitted model which 
includes predictors at level-2. 

Proportion reduction in variance or “variance explained” at level- 1 is 
computed as: 

Proportion of variance explained level 1 

6^ (model without predictors) - 6^ (model with predictors) 

6^ (model without predictors) 

The estimated variance in the model that did not include predictors at the 
level- 1 is (j 2 =260.90. Once the three predictors are included in the model, the 
estimate of student-level variance is now 156.37. Thus, proportion of 
variance at level 1 for this sample is: 

(247.84-156.37) 91.47 

— = = 0.369 

247.84 247.84 

It is clear that adding the three predictors reduced the within-department 
variance by 37%. Hence, it can be concluded that the three predictors accounts 
for about 37% of student-level variance in the motivation variable. This means 
that 37% of the variance in department mean motivation is accounted for by 
active learning strategies, higher-order thinking skills, and teacher student 
relations. 

Table 6 provides estimates of the variance components for the random 
effects and tests of hypothesis that these variance components are zero. For 
this sample, department mean, and active learning strategies slopes varied 
significantly. This means that for some departments, the slopes are much 
steeper than for other departments, that is, the relationship with motivation is 
much stronger in some departments than in other departments. The variability 
among departments also suggests that department-level variables might 
account for some of the differences. 

Research Question 4\ Do department level factors influence the magnitude 
of the student factors on motivation scores? 

The main effect for quality of teaching and learning was found to be 
significant across departments while that of student satisfaction with course 
contribution to their learning was not significant (see Table 5). Further, none 
of the department-level factors influenced the slopes of student level factors, 
that is when the three coefficients (level 1 regression coefficients) were 
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Table 6 



Hierarchical Linear Model Variance Components and Standard 
Deviation for Level-2 Predictors for Study Sample 



Random Effects 


Variance 

Component 


Standard 

Deviation 


Department mean (Intercept) 


9.74* 


3.12 


ALS 


0.01* 


0.09 


HOTS 


0.00 


0.05 


TSR 


0.00 


0.02 


Level- 1 Residual variance 


156.37 


12.50 



Note: * /? < 0.05. 



modeled with department-level variables. These results indicate that there 
were no cross-level interactions, meaning, a department-level variable (quality 
of teaching and learning and student satisfaction with course contribution to 
their learning) did not influence student-level slope. 

Discussion and Implications 

The purpose of this study was to examine factors within and across college/ 
university departments that college instructors, administrators and support 
staff can influence to increase student motivation and thus student success. 
Specifically, this study was guided by four research questions: a) Are there 
differences in motivation scores among university departments? b) Do 
department-level factors (quality of teaching and learning, student satisfaction 
with course contribution to their learning) explain the differences in mean 
department motivation scores? c) Do student factors (higher-order thinking 
skills, active learning strategies, and teacher student relations) explain 
differences in motivation scores? d) Do department- level factors influence 
the magnitude of the student factors on motivation scores? 

At the student-level, there were three primary independent variables 
(measures) of concern in these analyses; a) higher-order thinking skills; b) 
active learning strategies; and c) teacher student relations. The dependent 
variable was motivation. The department-level predictors are; a) quality of 
teaching and learning; and b) student satisfaction with course contribution to 
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their learning. The results of the study indicated significant variations among 
university departments in their motivation scores. This suggests that students 
were more variable in their motivation scores. At the department-level, the 
main effects for quality of teaching and learning was significant indicating 
that department motivation means are significantly influenced by quality of 
teaching and learning. This finding makes intuitive sense since departments 
that have clear policies of course structuring and development of clear learning 
objectives make it easier for learners to monitor their learning progress while 
remaining motivated throughout. Further, the reliability coefficients for 
department motivation means, the randomly varying slopes for active learning 
strategies and higher-order thinking skills were all moderate. Table 6 provides 
estimates of the variance components for the random effects and tests of 
hypothesis that these variance components are zero. Also, the department 
mean, and active learning strategies slopes varied significantly. This finding 
means that for some departments, the slopes are much steeper than for other 
departments, that is, the relationship with motivation is much stronger in 
some departments than in other departments. The variability among 
departments also suggests that department-level variables, such as quality of 
teaching and learning accounts for some of the differences. The results 
document the direct effects of the individual correlate and predictors (personal 
variables), and institutional variables (department variables for this study), 
on the measure of motivation. These core findings of the study and the 
statistical technique utilized have a variety of implications for higher education 
and practice, theory development, methodology and research design, and 
future research. 

Implications for Higher Education and Practice 

Understanding individual and institutional factors related to student 
motivation in learning are important professional issues. All the three student- 
level predictor variables (higher-order thinking skills, active learning 
strategies, and teacher student relations) have a rather rich history in the 
empirical and theoretical literatures pertaining to student-faculty interaction 
(Astin, 1993; Pascarella &Terenzini, 1991; and Tinto, 1987) and explanation 
of motivation (DiCintio & Stevens, 1997; House, 1997; Plecha, 2002; Tavani 
& Losh, 2003). 

The results of this study and those of other recent studies (Astin, 1993; 
DiCintio & Stevens, 1997; House, 1997; Kuh & Hu, 2001), clearly show that 
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higher-order thinking skills, active learning strategies, and teacher student 
relations are important variables in predicting motivation. Further, department- 
level variable of quality of teaching and learning is an important component 
to predicting motivation. Therefore, it seems important for university 
departments, colleges, schools, and university faculty to be sensitive to these 
student-level characteristics. Leaders in various units of colleges and 
universities behoove them to be cognizant of the elements of higher-order 
thinking skills, active learning strategies, teacher student relations, and quality 
of teaching and learning so that they are fully involved in providing the kinds 
of educational experiences that can enhance the development of these 
important and affective motivation characteristics in their learners. This 
strategy may lead to increase student motivation, elevated active learning 
strategies and development of higher-order thinking skills and thus student 
academic success. Strengthening these individual and institutional 
characteristics seems particularly important for learners and institutions of 
learning. 

Implications for Theory Development 

The results of this study have implications for the continued development 
of a nomological network (Cronbach & Meehl, 1955) for a general theory of 
motivation and learning in institutions of learning and other organizations. 
This research approach and concern has been on studying student-level and 
department-level factors related to motivation. The research believes this 
approach to theory development will provide a conceptual framework for 
future research that has stronger implications for improving strategies used 
to motivate learners. Thus, a developing theory of motivation and learning 
can be understood at both the individual student and organizational levels 
(departments). Obviously, there are a host of other organizational and personal 
variables that can be researched and added to the nomological network in 
explicating a theory of motivation and learning. With this goal in view, this 
study and other recent studies (e.g., Plecha, 2002; Tavani, & Losh, 2003) will 
shed more light to the current research on predictors and correlates of 
motivation. 

Implications for Future Research 

This study was a cross-sectional in nature, completed at only one point in 
time with one large, institution sample of traditional and non-traditional 
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students. Replications of the study, with the refined study measures resulting 
from the confirmatory factor analyses, and the addition of other important 
measures as well, are needed. For the most part, the measures used in this 
study yielded reliable data, though some of the measurement dimensions may 
need to be refined with revisions of items. The researcher believes these 
measures are adequate to do replication studies in other large research/ 
extensive doctoral university contexts, and with other research designs. The 
findings of this study suggest that these variables may be quite potent and 
yield rich information for theory development. As well, the continued use of 
mixed methodologies in future studies can strengthen the nomological network 
(Cronbach & Meehl, 1955) of a theory of motivation and learning and add to 
the utility and explanatory power of the quantitative results presented in this 
study. 

Further, the synthesis of the pertinent studies produced greater benefits 
that augmented the statistical results. Majority of past research have made 
use of final grades and student success as indicators of good teaching, learning 
and student motivation. This study advocates for the need for learner-centered 
research that focuses on the cognitive aspect of all learners; how they learn, 
how learning might be increased, and what environmental factors can assist 
in achieving such improved results and motivation. 

Since motivation consists not only of making student receptive to and 
excited about the subject taught, but also making them discern the value of 
learning itself (Covino, & Iwanicki, 1996), there are a number of strategies 
that are of value to the teacher in order to effectively motivate individual 
student or groups of students. First, through teacher student relations, the 
teacher gets to know students’ preconceptions and misconceptions or subject 
matter, student’s active learning strategies such as student-to- student relations 
and how such relationships could further the learning process, students’ areas 
of interests, student weak points, students’ ability to learn factual information 
and to develop concepts, understanding and applying principles, rules, 
applying theories, and problem solving strategies among others. With this 
knowledge, the teacher can devise strategies to foster motivation. Good and 
Brophy (1987) presented four areas such as supportive environment, as 
espoused in the elements of teacher student relations such as teacher taking 
a personal interest in student and student learning, considering student feelings, 
helping student when they are faced with trouble with work and maintaining 
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frequent communication. A second strategy is to provide an appropriate level 
of challenge or difficulty as listed in the elements of active learning strategies 
such as clarity with which the course objectives are communicated, use of 
class time, clarity with which student responsibilities and expectations are 
explained, teaching and learning techniques used during the course, quantity 
and quality of feedback provided on tests and graded work among others. A 
third strategy requires that the teacher provides meaningful learning objectives 
so that the student remains encouraged in expressing their own ideas, 
participate in small and large discussion groups, compare and contrast ideas, 
and appreciate to learn from each other. The fourth strategy involves 
moderation and variation in strategy such as dividing a class time into a variety 
of activities such as lecture, small groups, large group projects and 
presentation, and discussion groups. 

Further, subjects such as mathematics and science courses are quite often 
best taught using a teacher- centered style where the students are taught a 
particular skill and then asked to duplicate that skill on their own until mastery. 
Social sciences and humanities often are exactly the opposite, opening up 
much greater opportunities for in-class discussions, group projects, and 
extended peer interaction and differential influence on motivation and 
persistence. 

The findings of this study call for continued research on correlates and 
predictors of motivation. It must be acknowledged that there may be more 
variables affecting motivation that cannot be altered than those that can. 
Demographic, personality variables and family patterns, for example, may 
be the strongest predictors of motivation but lend very little to manipulation. 
This is not to say that researching these alterable variables is useless because 
of their relatively small influence. Rather, as educators, our noble task is to 
seek to influence what we can. Based on the findings of this study, it is evident 
that a systematic evaluation of correlates and predictors of student motivation 
at the university level requires multilevel and multi-measure approaches to 
the analysis of student motivation. Although the results of this study may not 
generalize to other universities, they are expected to inform us about desired 
data and methods for a more systematic approach to correlates and predictors 
of student motivation in institutions of higher learning. 

Based on the results of this study, faculty who wish to increase the level 
of student motivation in their classes should focus on improving the overall 
quality of their teaching, be sure to include elements of active learning 
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strategies and teacher student relations, and to create a classroom environment 
that encourages relationships with other students. Such changes in teaching 
methods are likely to increase motivation far more than increased emphasis 
on higher-order thinking skills. 

Information about how students perceive the quality of teaching and 
learning, the effectiveness/enhancement of their own learning, and important 
elements of the learning environment can provide a rich base for enhancing 
the quality of teaching and learning in higher education settings. 

Implications for Research Methodology 

Hierarchical linear modeling (HLM) statistical approach was utilized 
because of the hierarchical nature of the data and the nature of questions 
posed. HLM was conducted with three Level- 1 and two Level-2 predictors. 
A model without predictors in the model (unconditional model) indicated 
that seventeen percent (17%) variability in motivation was due to departments. 
With three predictors included in the Level- 1 model, the explainable variability 
in motivation rose to 37%. 

Further, it is important that a correct choice of research design, statistical 
technique and presentation of research findings continued to be emphasized 
as a foundation of good research. From the research design and statistical 
technique utilized here, it is clear that if standard regression analysis with 
individual student as a unit of analysis was utilized, it would have resulted in 
smaller standard errors and possibly incorrect conclusions. Further, if analysis 
of department-level variables at the individual, had been used, it would have 
resulted in aggregation bias. It is worth noting that aggregating student-level 
data to a higher hierarchy (department-level in this case) fails to fully capture 
the effects of outcome variables that may operate at both levels of analysis 
(Fullerton, 2002; Hox 2002; Singer, 1998; Snijders & Bosker, 2000). It is the 
research design and statistical technique that has been used in this study (HLM) 
that allow researchers to model individual outcomes within groups and then 
to identify and model any between-group differences that occur (Bryk & 
Raudenbush, 1992; Hox, 2002; Raudenbush & Bryk, 2002). 

For future research, it will be crucial that data is disaggregated to class 
level, discipline level, department level, or college level so that contribution 
of each level is assessed. Without breaking down the data to various levels 
stated above, it would be difficult to assess the effect size each level contributes 
to the differences in motivation. 
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As noted earlier, it would be useful to analyze this data broken down by 
academic level, type of course (introductory lecture, lab, senior seminar, etc.), 
and class size. Such factors may have more to do with all the constructs of 
teaching and learning considered in this study in relation to both student 
motivation and emphasis on higher order thinking skills. 

Limitations of the Study 

It should be noted that that generalization to populations comprised of 
traditional age college students must be viewed with caution as less than a 
quarter of the students in this sample were under the age of 21 and 40% 
worked full-time. In other ways, the sample resembled a fairly typical 
academic cross-section of undergraduate students at large public universities 
in the United States. It is important to note in consideration of higher-order 
thinking skills that the level and course content were not analyzed in this 
study due to limitation of the data. An introductory freshman course on world 
history may be expected to appropriately place less emphasis on higher-order 
thinking skills than a senior level history seminar on the transition from 
colonialism to independence in sub-Saharan Africa. The latter would probably 
attract mostly history majors who had a number of prerequisite courses and 
were thus prepared to be challenged with a strong emphasis on higher-order 
thinking skills. This study considers all levels of courses in a wide variety of 
academic departments but results may vary significantly in a focus on a 
particular course. 

This study does not differentiate results by level of course or whether 
the course is an elective, general education, or major requirement. Motivation 
and involvement and emphasis on higher-order thinking skills are likely to 
be perceived differently by students in large introductory lecture courses versus 
small seminars for majors only that may be senior capstone experiences. Class 
size is also not taken into account but has been shown to affect course 
evaluations as well as the way a course is taught. 
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APPENDIX A 



Eactors and Sample Items Operationalizing Each Eactor 
(Survey Adapted from Rugutt, Ellett, & Culross, 2004) 

Teacher Student Relations (TSR) 

• The teacher takes a personal interest in me. 

• The teacher considers my feelings. 

• The teacher helps me when I have trouble with the work. 

• The teacher talks with me. 

Higher Order Thinking Skills (HOTS) 

• Learning factual information. 

• Developing concepts. 

• Understanding and applying principles and rules. 

• Understanding and applying theories. 

• Critical analysis and/or problem solving. 

Active Learning Strategies (AES) 

• I make friendships with other students. 

• I know other students. 

• I do favors for members of this class. 

• Students help me with my learning. 

• I help other class members who are having trouble with their work. 

• In this class, I am able to depend on other students for help. 

• I explain my ideas to other students. 

• Students discuss with me how to go about solving problems. 

• I discuss different answers to questions. 

• I cooperate with other students when doing assigned work. 
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• I share my books and resources with other students when 

doing assignments. 

• I learn from other students in this class. 

• I work with other students in this class. 

• I cooperate with other students on class activities. 

• I work in groups in this class. 

Motivational (MI) 

• Encouragement for students to express their own ideas. 

• Encouragement for students to participate in discussions. 

• The extent to which students are encouraged to compare and contrast 
ideas. 

• The extent to which students are involved in discussions among 
themselves. 

• The extent to which students learn from one another. 

Student Satisfaction with Course Contribution to their Learning (SATISFY) 

• How would you grade the quality of teaching in this course? 

• What was the contribution of the course to your personal learning? 

• How would you grade this course overall? 

Quality of Teaching and Learning ( QTL) 

• Clarity with which the course objectives are communicated. 

• Clarity with which student responsibilities and expectations are 
explained. 

• Use of class time. 

• Outside assignments and integration of outside assignments with other 
course activities. 

• Teaching and learning techniques used during the course. 

• The instructor’s enthusiasm for teaching, learning and the subject 
taught. 

• The interpersonal climate in the classroom (e.g., patience, courtesy, 
respect). 

• Encouragement for students to express their own ideas. 

• Encouragement for students to participate in discussions. 

• Clarity and understandability of the instructor’s speech. 

• Directions and explanations given for course content. 

• The kind and number of thought-provoking questions asked. 



Fall 2004 / Volume 4, Number 2 



88 



• The extent to which students are encouraged to compare and contrast 
ideas. 

• The degree to which the instructor helps students organize information 
and understand relationships among various topics. 

• Explanation(s) given for difficult material/ideas. 

• Encouragement for students to ask questions. 

• Feedback about learning provided during teaching and learning 
activities. 

• The extent to which adjustments are made in a lesson when needed. 

• The degree to which students are encouraged to apply course content 
to solve problems or to understand real life situations. 

• The quantity/quality of feedback provided on graded work. 

• The quantity/quality of feedback provided on tests given. 

• The extent to which students are provided opportunities to determine 
their progress in the course. 

APPENDIX B 



Definitions of Study Terms and Variables 

Motivation : Motivation has been defined as the energy that is innate within 
all individuals, and high levels directed toward a particular situation results 
in greater amounts of energy expended on that task. The greater desire and 
higher levels of energy targeted at accomplishing a goal the higher the levels 
of performance (McDevitt & Ormrod, 2004). Operationally, the terms 
“motivation” and “motivation involvement” will be considered equivalent. 
This includes encouragement for students to express their own ideas, 
encouragement for students to participate in discussions, the extent to which 
students are encouraged to compare and contrast ideas, the extent to which 
students are involved in discussions among themselves, and the extent to 
which students learn from each other 

Higher Order Thinking Skills : In the literature review, higher order 
thinking skills are measured by tests of cognitive ability such as problem 
solving or integration of concepts (Kanfer & Ackerman, 1989). However, in 
this study the terms, “higher order thinking skills” and “emphasis on higher 
order thinking skills” are equivalent and refer to the operational definition 
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above. Higher order thinking skills involved student learning factual 
information, developing concepts, understanding and applying theories, and 
critical analysis and/or problem solving (Rugutt, Ellett & Culross, 2003). 

Student-Faculty Interaction: the contact between the faculty and the 
student. This includes course related activities and activities other than the 
course work/outside the course work (Chemosit, 2004). The measure that 
describes student-faculty interaction includes activities such as teacher taking 
a personal interest in student, teacher considering student’s feelings, teacher 
helping the student when he/she is trouble with the work, and teacher talking 
to with the student (Author). 

Active Learning Strategies: activities in which the students are actively 
involved or engaged or are required to take an initiative in enhancing their 
own learning. Active learning strategies include how often the student worked 
with other students, student knowing other students in class, student explaining 
his/her ideas to other students, student learning from other students, student 
cooperating with other students on class activities, working in groups in class, 
and being involved in other vigorous action (individual/joint) in pursuit of 
knowledge or skills (Chemosit, 2004). 
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